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54. Nighttime vehicle recognition device 
57. [Abstract] 

[PurposeJ To recognize other vehicles at night by detecting the lights emitted by them without erroneously 
recognizing street lights, etc. 

[Composition] When an image ahead of a vehicle is input (Step 601), the headlight switching device to which the 
nighttime vehicle recognition device according to the invention has been applied converts this image into binary data 
(Step 602), extracts outlines (Step 603), determines the sizes of bright areas (lights) (Step 6.04),' and based on this 
information, extracts a pair having symmetry (Step 605). In the image captured after a predetermined time, the 
symmetry of this pair is tracked. If it can be tracked, the lights are judged to be from light sources on another vehicle 
(Step 606) and the distance from that vehicle is determined (Step 607); and if the distance from that vehicle is 
decreasing, the other vehicle is judged to be oncoming, and the headlights are switched to the low beam (Step 608). 
Therefore, the headlights can be switched properly without recognizing street lights, etc. as light sources on other 
vehicles. 
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(START) 
I 

[Inputs an image.] - 601 
I 

[Converts the input image into binary data.] - 602 

[Extracts the outline of the image that has been converted into binary data.] - 603 

[Computes light size.] - 604 
I 

[Extracts light symmetry.] - 605 

- I ' 
[Tracks symmetry.] - 606 
I 

[Computes distance.] - 607 
I 

[Switches headlights.] - 608 



(STOP) 



[Claims] 



SSed wto ni§httime V6hiCle reC ° Sniti0n dCViCe that U 111313116(1 fa a Vehide » d te other vehicles at night, 



an image-capturing means that captures the image ahead of the aforementioned vehicle when it is movinc as horizontal 
bSX1fX~' bdghtaeSS faf0mati0n ^ diVi ^ g ' ' mt ° Predete ^ d ^'d~e 

l^-^Z^; ^ ** g£nerateS imagCS ° n th£ ^ htness captured by said 

an image storage area for storing the aforementioned binary images; 

an image storage means for storing the aforementioned binary images in the aforementioned image storage area; and 

nX^trT 0163113 eXamineS ** behaVi ° r ° f Cach Hght SOurce md det ™ s whether or not said each 
hght source belongs to another vehicle, by comparing the aforementioned two binary images stored m tne 

nZZ ^ St0rage afea ^ * e aforementioned ^ -ans at differences vvh le te veh" .e i 



[Claim 2] A nighttime vehicle recognition device according to Claim 1, further provided with 

a r h eaSU H? ment mCanS C ° mpUteS the horizontal-direction length and vertical-direction length, inside the ima*e of 
each light source extracted by the aforementioned light source extraction means; B ' 

diS h inn°," rCe th Paii ' §ener ! t !° n m ! anS CXtraCtS tW ° Hght S0UrCeS whose horizontal-direction length and vertical- 
d.rect,on length, computed by sa.d measurement means, are approximately equal, as a light source paired 
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a computation' means that computes the distance in the image, of the two light sources comprising the light source pair 
and/or the slope in the- image, of the line segment connecting the two light sources, for each of the light source pairs 
extracted by said light source pair generation means; wherein 

the aforementioned vehicle recognition means 

tries to match two aforementioned binary images based on the aforementioned distance and slope computed by the 
computation means, for the individual light source pairs extracted by said light source pair generation means, and 
determines whether or not said light source pair is coming from light sources on another vehicle depending on whether 
or not matching can be established. 

* 

[Claim 3] A nighttime vehicle recognition device according to Claim 2 wherein 
the aforementioned light source pair generation means 

extracts the aforementioned light sources by first extracting one of the aforementioned light sources, and next by 
searching for a second light source that satisfies the conditions that the slope of the line segment connecting said second 
light source to said first light source is approximately horizontal and also that the distance from said second light source 
to said first light source is within the value that corresponds to the vertical-direction length or horizontal-direction 
length measured by the aforementioned measurement area for said first light source. 

[Claim 4] A nighttime vehicle recognition device according to Claim 2 or 3 wherein 

the aforementioned vehicle recognition means 

treats the mid-position between the two light sources comprising the light source pair as the position of said light source 
pair; 

sets up a rectangular area having the vertical and horizontal sizes that correspond to the distance between the two light 
sources comprising said light source pair, using the position of the light source pair in one of the aforementioned binary 
images as the reference; and 

uses only the binary images inside the aforementioned rectangular area when comparing the aforementioned two binary 
images in order to examine the behavior of the aforementioned light source pair. 

[Claim 5] Any of the nighttime vehicle recognition devices according to Claims 2 through 4, further provided with 

an inter-vehicle distance computation means that computes the distance between said other vehicle and the vehicle on 
which said nighttime vehicle recognition device is installed, based on the distance between the light sources in said light 
source pair computed by the aforementioned computation means, wherein 

the aforementioned vehicle recognition means determines 

whether or not the light source pair judged to be from light sources on another vehicle belongs to an oncoming vehicle 
or the preceding vehicle, based on the change in the inter-vehicle distance computed by the inter-vehicle distance 
computation means. 
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[Detailed explanation of the invention] 

[0001] 

[Industrial field of application] This invention relates to a device that is installed in a vehicle and that detects other 
vehicles at night. 

[0002] 

[Prior art] Devices have been suggested that automatically switch the angles of headlights at night when an oncoming 
vehicle is passing by. In such devices, the approach of the oncoming vehicle must be accurately detected. For 
example, JP Sho. 61-285153 shows a device that has light-receiving elements arranged horizontally on the front of a 
vehicle, and that detects an oncoming vehicle based on the variability among the amounts of light detected by the 
individual light-receiving elements. According to this device, when the oncoming vehicle comes closer, the light- 
receiving element that is squarely facing the other vehicle's headlights receives a large amount of light. On the other 
hand, those light-receiving elements that are farther from said light-receiving element receive significantly less light 
since they have larger angles relative to the illumination angle of the light, thus creating substantial variability among 
the amounts of light detected by the individual light-receiving elements. In contrast, when the light-receiving elements 
.receive the light from the vehicle's own headlights reflected from guardrail, etc., the amounts of light received by the 
individual light-receiving elements are nearly uniform, and therefore an oncoming vehicle can be detected. 

[0003] 

[Problems that the invention is to solve] However, according to the aforementioned device, the amounts of light 
received will vary greatly also when the light is coming from streetlights, decorative lights, etc., thus erroneously 
recognizing streetlights, etc. as other vehicles. In view of this , problem, the objective of the present invention is to 
recognize the light emitted by other vehicles at night without recognizing street lights, etc. 

[0004] 

[Means of solving the problem] The invention described in Claim 1 of the present invention developed in order to 
solve the aforementioned problem is a vehicle recognition device that is installed in a vehicle and that detects other 
vehicles at night, provided with an image-capturing means that captures the image ahead of the aforementioned vehicle 
when it is moving, as horizontal and vertical two-dimensional brightness information by dividing it into predetermined 
pixels and by detecting the brightness of each pixel; a light source extraction means that generates binary images based 
on the brightness information captured by said image-capturing means; an image storage area for storing the 
aforementioned binary images; an image storage means for storing the aforementioned binary images in the 
aforementioned image storage area; and a vehicle recognition means that examines the behavior of each light source 
and determines whether or not said each light source belongs to another vehicle, by comparing the aforementioned two 
binary images stored in the aforementioned image storage area by the aforementioned image storage means at different 
times while the vehicle is moving. 

[0005] The invention according to Claim 2 is the nighttime vehicle recognition device according to Claim 1, further 
provided with a measurement means that computes the horizontal-direction length and vertical-direction length, inside 
the image, of each light source extracted by the aforementioned light source extraction means; a light source pair 
generation means that extracts two light sources whose horizontal-direction length and vertical-direction length, 
computed by said measurement means, are approximately equal, as a light source pair; and a computation means that 
computes the distance in the image, of the two light sources comprising the light source pair and/or the slope in the 
image, of the line segment connecting the two light sources, for each of the light source pairs extracted by said light 
source pair generation means; wherein the aforementioned vehicle recognition means tries to match two 
aforementioned binary images based on the aforementioned distance and slope computed by the computation means, 
for the individual light source pairs extracted by said light source pair generation means, and determines whether or not 
said light source pair is coming from light sources on another vehicle depending on whether or not matching can be 
established. 
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[0006] The invention according to Claim 3 is the nighttime vehicle recognition device according to Claim 2 wherein 
the aforementioned light source pair generation means extracts the aforementioned light sources. by first extracting one 
f of the aforementioned light sources, and next by searching for a second light source that satisfies the conditions that the 
slope of the line segment connecting said second light source to said first light source is approximately horizontal and 
also that the distance from said second light source to said first light source is within the value that corresponds to the 
vertical-direction length or horizontal-direction length measured by the aforementioned measurement area for said first 
light source. 

[0007] The invention according to Claim 4 is the nighttime vehicle recognition device according to Claim 2 or 3 
wherein the aforementioned vehicle recognition means treats the mid-position between the two light sources 
comprising the light source pair as the position of said light source pair; sets up a rectangular area having the vertical 
and horizontal sizes that correspond to the distance between the two light sources comprising said light source pair, 
using the position of the light source pair in one of the aforementioned binary images as the reference; and uses only the 
binary images inside the aforementioned rectangular area when comparing the aforementioned two binary images in 
order to examine the behavior of the aforementioned light source pair. 

[0008] The invention according to Claim 5 is a nighttime vehicle recognition device according to any of Claims 2 
through 4, that is further provided with an inter- vehicle distance computation means that computes the distance between 
said other vehicle and the vehicle on which said nighttime vehicle recognition device is installed, based on the distance 
between the light sources in said light source pair computed by the aforementioned computation means, wherein said 
vehicle recognition means determines whether or not the light source pair judged to be from light sources on another 
vehicle belongs to an oncoming vehicle or the preceding vehicle, based on the change in the inter-vehicle distance 
computed by the inter-vehicle distance computation means. 

[0009] 

[Operation and effects of the invention] In the nighttime vehicle recognition device described in Claim i, an image- 
capturing means captures the image ahead of the vehicle as vertical-horizontal two-dimensional brightness information. 
This brightness information becomes a binary image after being separated into bright areas and dark areas by a light 
source extraction means, and is stored in an image storage area by an image storage means. By comparing two such 
binary images that have been stored in this way, it is possible to determine whether or not bright areas (i.e., light 
sources) exist and their behaviors. Based on this comparison result, the vehicle recognition means can determine 
whether or not the light source belongs to another vehicle. 

[0010] That is, because the nighttime vehicle recognition device described in Claim 1 takes in the brightness 
information that is ahead of the vehicle as vertical-horizontal 2-dimensional information, i.e., plane Information, it can 
recognize the position of the light source within that plane in both the horizontal and vertical directions. In other words, 
multiple light sources that would be interpreted by a device having light-receiving elements arranged horizontally as 
being the same because their positions in the horizontal direction are identical, can be individually recognized because 
of the differences in their positions in the vertical direction. 

[0011] Moreover, because the image storage means stores the aforementioned brightness information that has been 
converted into a binary image by the light source extraction means in the image storage area, positional changes in light 
sources can be examined by comparing two binary images that are stored at different times. By performing this 
examination, the vehicle recognition means can determine whether or not the light source belongs to a vehicle. For the 
determination method, a method such as that described below can be used, for example. 

[0012] That is, light sources that exist ahead of a vehicle mainly include the headlights of oncoming vehicles, the tail 
lamps of the preceding vehicles, streetlights, etc. The speed of each of these light sources relative to a vehicle equipped 
with said nighttime vehicle recognition device when the vehicle is moving, is normally the greatest for the headlights of 
the oncoming vehicles, followed by streetlights and the tail lamps of the preceding vehicles, in that order, and 
consequently, the severity of their behaviors in the image also follows the same order. Therefore, by evaluating the 
severity of their behaviors, i.e., the magnitudes of positional changes in these light sources, these three types of light 
sources can be differentiated from each other. Furthermore, because the image-capturing means has taken in the image 
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ahead as 2-dimensional brightness information, not only the behaviors of the light sources but also light source sizes 
(e.g., vertical-direction length and horizontal-direction length, or area, etc.) can be evaluated. Based on these sizes, 
individual light sources can be differentiated from each other on the same screen, or corresponding light sources can be 
identified to be the same between the aforementioned two binary images. 

[0013] In the nighttime vehicle recognition device described in Claim 2, the measurement means computes the 
vertical-direction length and the horizontal-direction length inside the image, of the light sources extracted by the light 
source extraction means. Then, the light source pair generation means extracts two light sources having approximately 
the same values as the aforementioned computed values as a light source pair, and the computation means computes the 
distance, between the two light sources comprising each of the extracted light source pairs and the slope of the line 
segment connecting the two light sources, inside- the image. 

[0014] The vehicle recognition means tries to match the aforementioned two binary images based on the 
aforementioned distance and slope computed by the computation means, for the individual light source pairs extracted 
by said light source pair generation means, and determines whether or not said light source pair is coming from light 
sources on another vehicle depending on whether or not matching can be established. . 

[0015] That is, in the nighttime vehicle recognition device described in Claim 2, the light sources stored in the image 
storage area are extracted as pairs, and therefore, those light sources that do not make up a pair, such as streetlights, are 
■excluded from extraction targets, making the process of matching between binary images more efficient. If a light 
source pair belongs to a vehicle, the aforementioned distance and slope should remain roughly the same between two 
binary screens [sic] taken within a short interval of time, and therefore, by trying to match two binary screens [sic] 
based on these distances and/or slopes, it is possible to determine whether or not the light source pair belongs to a— 
vehicle. 

[00 16] During said matching attempt, using only the distances between light source pairs can prevent light source pairs 
having excessively long (or short) distances from being judged to be light source pairs belonging to vehicles. Likewise, 
using only the slopes can prevent light source pairs having excessively large slopes from being judged to be light source 
pairs belonging to vehicles. Using both distance and slope can exclude all light source pairs exhibiting at least one of 
the aforementioned inappropriate values from the determination targets, thus improving the determination reliability. 

[0017] Furthermore, in the nighttime vehicle recognition device described in Claim 3, the light source pair generation 
means extracts a light source pair by first extracting a light source, and next by searching a second light source using 
said first light source as the reference. Moreover, this search is performed under conditions that the slope of the line 
segment connecting said second light source to the light source being used as the reference is approximately horizontal 
and that the distance from said second light source to said first light source is within the value that corresponds to the 
vertical-direction length or horizontal-direction length measured by the aforementioned measurement area for said first 
light source. Therefore, those light sources for which the slope of the line segment connecting two light sources is too 
large for light sources belonging to a vehicle and for which the distance between two light sources is too large for the 
light sources to belong to a vehicle compared to the sizes of the two light sources, are not extracted as light source pairs. 

[0018] That is, in the nighttime vehicle recognition device described in Claim 3, only those light sources satisfying the 
aforementioned conditions are extracted during light source search instead of searching for light sources from the entire 
image, and therefore those light sources that are inappropriate as light sources on vehicles are excluded, thus reducing 
the number of light source pairs to be extracted. Consequently, matching of light sources that is attempted between two 
binary screens after light source pairs have been extracted, can be performed more efficiently. 
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[0019] Furthermore, when checking the match between a light source pair in two binary images, the nighttime vehicle 
recognition device described in Claim 4 sets up a rectangular area having the vertical and horizontal sizes that 
correspond to the distance between the two light sources comprising said light source pair, in the same position on the 
other binary image, using the position of the light source pair in one of the aforementioned binary images as the 
reference, and searches for a light source pair that corresponds to the aforementioned reference light source pair. Note 
that the position of a light source pair is represented by the mid-position between the two light sources comprising the 
light source pair. 

■ 

[0020] That is, in the nighttime vehicle recognition device described in Claim 4, light source pairs whose positions are 
different enough to be outside the aforementioned area are excluded from search targets, and therefore determination of 
whether or not a light source pair belongs to a vehicle can be performed even more efficiently and reliably. 

[0021] In the nighttime vehicle recognition device described in Claim 5, the inter-vehicle distance computation means 
computes the distance between said other vehicle and the vehicle on which said nighttime vehicle recognition device is 
installed, i.e., the so-called inter-vehicle distance, based on the distance between the light sources in said light source 
pair computed by the computation means. The change in the inter-vehicle distance between two binary screens is then 
determined for the light source pair judged to be from light sources on another vehicle by the vehicle recognition 
device, and whether or not that light source pair belongs to an oncoming vehicle or the preceding vehicle. That is, if 
this distance is shrinking, the light source is judged to belong to an oncoming vehicle; and if this distance does not 
change much, the light source is judged to belong to a preceding vehicle. 

[0022] In other words, according to the nighttime vehicle recognition device described in Claim 5, the inter-vehicle 
distance computation means determines the distance to each light source pair judged to belong to another vehicle, and~- 
thus an oncoming vehicle can be differentiated from a preceding vehicle based on the change in this distance. 

[0023] 

[Working example] A working example of the invention will be explained with references to drawings. Figure 2 is a 
schematic configuration diagram showing the overall configuration of a headlight angle automatic switching device 1 to 
which the nighttime vehicle recognition device according to the invention has been applied. 

[0024] Headlight angle automatic switching device 1 consists of a camera 1 1 which acts as the image-capturing means 
of the invention for capturing the image 5 ahead of vehicle 3, ECU 13 which performs nighttime vehicle recognition, 
vehicle headlights 15, and actuators (not shown in the figure) which switch headlights 15. 

[0025] Figure 3 shows the internal configuration of ECU 13. ECU 13 consists of a CPU 21, a CPU bus 23, a binary 
threshold value generation circuit 25, a comparator 29 which converts an image signal 27 coming from camera 1 1 into 
binary data based on the threshold value and generates a binary image, an image bus 31, binary image memory 33 
which acts as the image storage area of the invention for storing binary images, 3x3 filter operation LSIs 35 for 
extracting the outlines of the binary images, 3x3 filter data ROM 37 for storing the computation data for 3 x 3 filter 
operation LSIs 35, RAM 39 for storing the program data for a digital signal processor (DSP), and a digital signal 
processor (DSP) 41. 

[0026] Note that headlight switching signal 43 is sent to a headlight switching area (not shown in the figure) via CPU 
bus 23, and this signal is used for switching between the main beam and the low beam (see Figure 2). The processing 
performed by this device will be explained using the flow chart in Figure 4 and the diagram in Figure 5. Note that 
Figure 4 visually illustrates how a binary image stored in binary image memory 33 is processed. This process is started 
by a known timer interrupt with a predetermined time interval. 
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[0027] First, an image is input in Step 601. In other words, an image signal from camera 1 1 in Figure 2 is input as 
picture signal 27 in Figure 9. Next, the input image is converted into binary data in Step 602. This is accomplished by 
inputting picture signal 27 into the input pin on the non-inverting side of comparator 29 in Figure 3, by creating binary 
picture signal 47 by inputting binary threshold value signal 45 generated by binary threshold value generation circuit 25 
into the inverting side of comparator 29, and by storing this signal as a binary image in binary image memory 33 via 
image bus 3 1. That is, binary threshold value generation circuit 25 and comparator 29 correspond to the light source 
extraction means of the invention, and the step of storing signals in binary image memory S3 corresponds to the 
processing performed by the image storage means. This binary image is shown as image 101 in Figure 5. 

[0028] Next, the outlines of the binary images are extracted in Step 603. This is accomplished by having 3 x 3 filter 
operation LSIs 35 in Figure 3 access via image bus 31, the binary images stored in binary image memory 33, and 
extract the outlines of the binary images based on the filter data stored in 3 x 3 filter data ROM 37. The outline images 
of the extracted binary images are stored in RAM 39 for DSP via image bus 31. These outline images are shown as 
image 102 in Figure 5, and the bright areas in image 101 are surrounded by outlines. Note that these areas surrounded 
by outlines (i.e., the bright areas in image 101) indicate headlights of oncoming vehicles, tail lamps of preceding 
vehicles, streetlights, etc., which will hereafter be summarily referred to as "lights." 

[0029] Next, computation of light sizes based on the outline images, i.e., the processing by the measurement means of 
the invention, is performed in Step 604. This is computed by having DSP 41 in Figure 3 process the outline images 
stored in RAM 39 for DSP. That is, light sizes and vertical and horizontal lengths shown in image 103 in Figure 5 are 
computed. These vertical and horizontal lengths are stored in RAM 39 for DSP. 

[0030] Next, light symmetry is extracted in Step 605. This is computed by DSP 4 1 based on the vertical and horizontah 
lengths stored in RAM 39 for DSP in Figure 3, which corresponds to the processing by the light source pair generation 
means of the invention. This is illustrated in image 104 in Figure 5. 

[003 1] Next, symmetry is tracked over two binary images in Step 606. This corresponds to the process of attempting 
to match light source pairs performed by the vehicle recognition device of the invention. This is computed by DSP 41 
based on the pairing data stored in RAM 39 for DSP in Figure 3 (i.e., data indicating which light is paired up with 
which light). This is illustrated in image 105 in Figure 5. 

[0032] Next, distance computation is performed in Step 607. This is back-calculated from the apparent width of a 
symmetric pair in the image by taking into consideration the magnification of the lens system used for taking the image. 
With this method, an accurate distance cannot be determined unless the true width of the headlights (or tail lamps) of 
the vehicle that produced the pair in the image is known. However, by back-calculating using the width of 
representative headlights (or tail lamps), an approximate distance can be determined. This method is sufficiently 
practical since the device for switching headlights does not require accurate distances. 

[0033] Next, headlights are switched in Step 608. If the approximate distance computed in Step 607 is equal to or 
shorter than a predetermined value (e.g., 1 00 m), the distance to an oncoming vehicle or the preceding vehicle is judged 
to be short, and the headlights are switched from the main beam to the low beam, and the process is finished. 

[0034] The details of symmetry extraction in the aforementioned Step 605 will be explained using the flow chart in 
Figure 6 and Figure 7 which visually illustrates the processing content. Note that, in order to simplify the explanation, 
light 49 (see Figure 7) only is used as the reference light, and the explanation will be limited to the process of searching 
for a light that forms a pair with light 49. 
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[0035] First, in Step 701, the larger of the horizontal-direction length Lx (see Figure 7) or the vertical-direction length. 
Ly (see Figure 7) of each light is designated as L. In the succeeding Step 702, the horizontal direction of the search 
range for the light that forms a pair with light 49 is specified, and the width of the search range is set in correspondence 
to the value of the aforementioned L. Here, this width is set to K-L by multiplying L by a proportional constant K, and 
is allocated to both sides of the center position of light 49 as shown in Figure 7. Note that the center position of a light 
is defined as the intersection between the line bisecting the vertical-direction length of the light and the line bisecting 
the horizontal-direction length of the light, and the position of each light is represented by this center position. 

[0036] Next, in Step 703, an angle area is determined in order to specify the vertical direction of the search range for 
the light that forms a pair with light 49. This angle area is selected so as to make the line segment connecting light 49 
with the paring light approximately horizontal, and is determined by specifying up and down angle tolerances from the 
horizontal direction with light 49 at the center. Here, as shown in Figure 7, an angle areas of 8 degrees up and down, 
i.e., a total of 16 degrees, are specified on both the left and right sides of light 49. Angle areas thus determined and the 
search range specified in Step 702 produce two search ranges 51a and 51b (see Figure 7) consisting of triangles. 

[0037] Next, in Step 704, lights whose center positions are located inside search ranges 51a and 51b are extracted. 
Here, in Figure 7, light 53 whose center position is located inside the search range of light 49 is extracted. When light 
53 has been extracted, the succeeding Step 705 checks each light's horizontal-direction size, i.e., the horizontal length 
determined in Step 604. If light 49 and light 53 are a pair coming from light sources on the same vehicle, the horizontal 
length Lx of light 49 and the horizontal length Lx' of light 53 (see Figure 7) should be nearly identical, and therefore 
this%hecking can determine whether or not the pair is from light sources on a vehicle. Here, proportional constants 
KLrxmax and KLrxmin are set up, and whether or not KLrxmax-Lx > Lx' > KLrxmin-Lx is satisfied is checked. 

[0038] Next, each light's vertical-direction size, i.e., the aforementioned vertical length, is checked in the same way in 
Step 706. That is, for the vertical length Ly of light 49 and the vertical length Ly' of light 53 (see Figure 7), whether or 
not KLrymax-Ly > Ly' > KLrymin-Ly is satisfied is checked (where KLLrymax and KLrymin are proportional 
constants). 

[0039] Next, an other light presence prohibition area is set up in Step 707. If light 49 and light 53 are a pair from light 
sources on a vehicle, a third light cannot be present between these two lights. Therefore, other light presence 
prohibition area 55 (see Figure 7) is set up, and if another light is present in this area, light 49 and light 53 are not 
considered a pair (Step 708). 

[0040] When all of the above check items have been satisfied, light 49 and light 53 are extracted (in Step 709) as a 
candidate pair from light sources on the same vehicle (hereafter simply referred to as "pair of lights"), and said 
symmetry extraction process is terminated. Next, the details of the symmetry tracking in the aforementioned Step 606 
will be explained according to the flow chart in Figure 8 and Figure 9 which visually illustrates the processing content. 
Note that symmetry tracking means the examination of where the pair of lights extracted in the aforementioned Step 
605 have moved to in the next image 59 (see Figure 9). The next image 59 is a binary image that is obtained by 
performing the same processes as in the aforementioned Steps 601 through 603, and that is delayed by the amount of 
time required for the above-listed processes, from the current image 57 (see Figure 9) from which symmetry was 
extracted. 

[0041] First, to quantify the characteristics of the pair of lights that have been extracted, distance W between the center 
positions of the two lights on the screen (see Figure 9) and the angle 0 formed by the horizontal line and the line 
segment connecting the center positions (see Figure 9) are computed (Step 801). The vertical-direction length and the 
horizontal-direction length that have already been measured in the aforementioned Step 604 are also used as quantities 
indicating the characteristics of this pair. Hereafter, for the lights comprising this pair, the horizontal- and vertical- 
direction lengths of the right-side light will be denoted as Lrx and Lry, respectively, and the horizontal- and vertical- 
direction lengths of the left-side light will be denoted as Llx and Lly, respectively (see Figure 9 for all). In other words, 
the processing performed in Step 801 corresponds to the processing performed by the computation means of the 
invention. 
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■ [0042] Next, a horizontal-direction search range is computed in Step 802. This step specifies the horizontal-direction 
size of the search range to be used for searching in the next image 59 for the lights that correspond to the extracted pair 
of lights. If that' pair is a light source pair, the wider its width W is, farther it should move in the next image 59. 
Therefore, using a proportional constant of Kx', Kx'-W is allocated with the mid point (x, y) between the two lights as 
the center and is -specified as the horizontal-direction search range on the next image 59. 

[0043] In the succeeding Step 803, the vertical-direction search range is computed in the same manner as in Step 802. 
That is, using a proportional constant of Ky\ Ky'-W is allocated with the mid point (x, y) as the center and is specified 
as the vertical-direction search range on the next image 59. Next in Step 804, the search range in the next image 59 is 
corrected by taking up-down and left-right movements into consideration. When camera 1 1 is mounted on a vehicle, it 
is presumed that pitching and rolling of vehicle 3 shift the positions of light source pairs within the screen for each 
image. Therefore, by denoting the up-down and left-right movements of a pair within the image as Ay and Ax, 
respectively, Kx'-W + Ax and Ky'-WAy [sic] are specified as the new horizontal-direction search range and the new 
vertical-direction search range, respectively. When both the horizontal- and vertical-direction search ranges have been 
specified in Steps 802 through 804 as explained above, a rectangular search range 61 as shown in Figure 9 is generated. 

[0044] Next in Step 805, a pair in the next screen is extracted, whose mid-point between two lights is positioned in 
search range 61 in the next image 59. For example, in Figure 9, a pair whose mid-point is located at coordinates (x\ y') 
inside search range 61 is extracted. For a pair extracted in this way, quantities that indicate its characteristics are 
extracted (Step 806). That is, using the same process as that used in Step 801, distance W between the centers and 
slope 9' (see Figure 9 for both) are computed, and the right-side light's horizontal-direction length Lrx' and vertical- 
direction length Lry' as well as the left-side light's horizontal-direction length Llx' and vertical-direction length Lly' are 
computed at the same time (see Figure 9 for all). 

[0045] Next, the size of the right light is checked in Step 807. If the pair extracted in the next image 59 and having its 
mid-point at (x', y') is what the light source pair having its mid-point at (x, y) in the current image 57 becomes after a 
move, the size of the right light should not change abruptly. For example, by setting up proportional constants 
KLrxmax and KLrxmin for the horizontal direction, whether or not KLrxmax-Lrx > Lrx' > KLrxmin-Lrx is satisfied is 
checked. Likewise, whether or not the vertical-direction is within the tolerance is also checked. 

[0046] Next, the size of the left light is checked in Step 808. This is based on the same concept as that used for 
checking the size of the right light; and for the vertical direction, proportional constants KLrymax and KLrymin are set 
up, and whether or not KLrymax-Lry > Lry' > KLrymin-Lry is satisfied is checked. Likewise, whether or not the 
horizontal direction is within the tolerance is also checked. 

[0047] Furthermore, the slope of the pair is checked in Step 809. If the pair extracted in the next image 59 and having 
its mid-point at (x\ y') is what the pair extracted in the current image 57 and having its mid-point at (x, y) becomes after 
a move, the slope of the pair should not change abruptly. Therefore, by setting up proportional constants KGmax and 
KGmin, whether or not K9max-0 > G' > KGmin-G is satisfied is checked. 
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[0048] A pair that satisfies all the checks in Steps 807 through 809 is selected as a pair of light sources belonging to the 
same vehicle (Step 810), and this process of tracking light symmetry is terminated. Figure 10 shows an example of 
switching headlights 15 according to the processes that have been explained along Figures 4 through 9. This figure 
shows the process in which the headlights, that have been switched to the low beam when an oncoming vehicle was 
approaching, are switched to the main beam when the oncoming vehicle is no longer detected; and when an oncoming 
vehicle is detected again and its distance from the vehicle in which said vehicle recognition device is installed reaches 
approximately 170 meters, the headlights are switched to the low beam again. 

[0049] Because the aforementioned method can obtain the brightness information that is ahead of the vehicle as an 
image, i.e., 2-dimensional data, in "Step 601, the sizes of the lights appearing as bright spots' on this image can be 
computed in Step 604. The sizes of lights thus computed are checked for approximate matching in Steps 705 and 706, 
and therefore streetlights, decorative lights, etc. whose bright areas have different sizes are excluded. 

[0050] Moreover, during the pairing attempt, a second light is searched for after defining (in Step 702) a horizontal- 
direction search range based on the size of light 49 which is used as the reference, and therefore two lights that are too 
far apart to belong to light sources on another vehicle are excluded from search targets, thus making the search more 
efficient. 

[0051] Furthermore, since angle areas are defined in Step 703 to require that the slope of the line segment connecting 
two lights be approximately horizontal, lights are excluded if their slope of the aforementioned line segment is too large 
to belong to light sources on another vehicle (Steps 705 and 706), thus making the search more efficient. Therefore, 
even if a bright area formed inside the image by a streetlight, etc. happens to have the same size as a light from another 
vehicle, this search range setting will most likely exclude such streetlight, etc. 

t 

[0052] Moreover, because presence prohibition area 55 is set up (in Step 707) and light sources that has another light in 
this area are excluded (in Step 708), there is already an extremely high probability at this stage that the two lights 
belong to light sources on another vehicle. This pair having an extremely high probability of belonging to light sources 
on another vehicle is tracked in the next image 59, and its behavior is examined (in Step 606). Therefore, even if a 
bright area formed by a streetlight, etc. has cleared the aforementioned checks after the processes through Step 605 
have been completed, that bright area can be excluded by examining its behavior. 

[0053] For example, suppose that a bright area formed inside an image by a streetlight happens to have approximately 
the same size as a bright area formed by a light from another vehicle, that the distance to this light happens to be also 
appropriate as a distance between headlights or tail lamps of a vehicle, that the angle formed by the line segment 
connected to said light also happens to be approximately horizontal, and that the streetlight is extracted as one of the 
light comprising a pair of lights, as a result. Even in such a case, the streetlight exhibits a behavior that is completely 
different from those of other vehicles' lights, and thus will not be regarded as a light from another vehicle. An example 
of a completely different behavior is provided below. For example, a pair of lights from light sources on another 
vehicle will move inside the image while maintaining the slope between the two light sources approximately horizontal 
in the next image 59 as well. In contrast, a streetlight is moving at a different relative speed from a vehicle, and thus the 
aforementioned slope gradually increases. Consequently, the streetlight will be caught by the check in Step 809 and 
can be excluded. 

[0054] Tracking of light source pairs in the next image 59 is carried out by searching for the mid-point of the two lights 
comprising a light source pair. Since search range 61 is set up also for this search, tracking can be performed 
efficiently. That is, search range 61 is defined in the horizontal and vertical directions in Steps 802 and 803, 
respectively, in correspondence to distance W between the two light sources comprising the light source pair, and 
therefore, the allowable movement range in the next image 59 can be set up according to the size of the light source 
pair. Consequently, those light source pairs that move too far can be excluded from search targets. 

[0055] Moreover, since left-right movements Ax and up-down movements Ay caused by pitching and rolling of 
vehicle 3 have been incorporated into this search ranges 61, searching can be performed even if vehicle 3 rocks. That 
is, headlight angle automatic switching device 1 determines whether or not lights belong to other vehicles based on the 

" InterLingiia 

(3 1 0)792-3 636/Fax(3 10)792-3642 
II 




r 



sizes of the lights ahead of vehicle 3, the distance between two lights, whether or not the line segment connecting the 
two lights is horizontal, as well as" the behaviors of the two lights inside images,.and thus does not mistakenly recognize 
streetlights, decorative lights, etc. and does not cause headlights 15 to be switched too frequently. 

[0056] An explanation has been provided above for headlight angle automatic switching device 1 to which the 
nighttime vehicle recognition device according to the invention has been applied. However, the invention is not limited 
in°its application to the aforementioned example and can be implemented in various formats. For example, the 
invention was applied to headlight angle automatic switching device 1 in the aforementioned working example, but can 
also be applied to a device that warns of an approaching oncoming vehicle. That is, it is possible to configure a device 
that alerts the driver using sound, etc. when an oncoming vehicle is detected. Such a device would make nighttime 
driving safer. 

[0057] It is also possible to configure a device that detects the distance to the preceding vehicle and alerts the driver 
using sound, etc. when that distance becomes too short. Such a device would make nighttime driving safer. 
Furthermore, if Wc is defined as the apparent width of the target pair on the screen (Llx/2) + W + (Lrx/2), the rate of 
change in Wc per unit time, dWc/dt, is proportional to the relative speed of the pair to the image-capturing position of 
the camera. Therefore, if Kw is defined as the proportional constant determined by the magnification of the camera 
system, Kw-dWc/dt becomes the speed of the vehicle measured from the installation position of the camera. This can 
be utilized by a nighttime vehicle speed measurement device. 

[0058] In the aforementioned working example, it was explained that the next image 59 is input anew in order to track 
the lights that form a pair. However, instead of using such a method, it is also possible to input the image using only 
the process that is equivalent to the aforementioned Step 601. In such a case, no symmetry tracking is performed for 
the image that is input first, for example, and symmetry tracking can be performed for subsequent images by comparing 
them to the previously input image. 

[0059] The processes in Steps 807 through 809 check the extracted pairs,_and it is also possible to additionally check 
the distance between the light sources that comprise a pair. For example, by defining W as this distance and KWmax 
and KWmin as proportional constants, whether or not KWmax-W > W > KWmin-W is satisfied is checked. Such a 
process will eliminate mistaken recognition of those light sources that satisfy the checks in Steps 807 through 809 but 
whose W is too wide to belong to light sources on a vehicle. 

[Brief explanation of drawings] 

[Figure 1] A block diagram illustrating the nighttime vehicle recognition device according to the invention. 

[Figure 2] A schematic configuration diagram showing the overall configuration of a headlight angle automatic 
switching device 1 which is a working example of the invention. 

[Figure 3] A diagram illustrating the internal configuration of ECU 13 of the working example. 

[Figure 4] A flow chart illustrating the processes performed by ECU 13 of the working example. 

[Figure 5]- A diagram visually showing how the binary image stored in binary image memory 33 is changed by the 
processing performed by ECU 13 of the working example. 

[Figure 6] A flow chart illustrating the light symmetry extraction process in the working example. 

[Figure 7] A diagram explaining the symmetry extraction process in the working example. 
[Figure 8] A flow chart illustrating the symmetry tracking process in the working example. 

[Figure 9] A diagram explaining the symmetry tracking process in the working example. 
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[Figure 10] A graph showing an example of how oncoming vehicles are recognized and how headlights 15 are 
switched by headlight angle automatic switching device I of the working example. 

[Explanation of symbols] 

I ... Headlight angle automatic switching device 
3. ..Vehicle 

5 ... Image ahead 

II ... Camera 
13... ECU 

.15 ... Headlights 
21... CPU 

33 ... Binary image memory 
49 ... Reference light 
51a, 51b ... Search ranges 
53 ... Light that has been found 
55 ... Presence prohibition area 
57 ... Current image 
59 ... Next image 
61 ... Search range 
101 ~ 106 ... Images 

[Figure 1] 1 

#1 Image-capturing means 

#2 Light source extraction means 

#3 Image storage means 

#4 Image storage area 

#5 Vehicle recognition means 



[Figure 2] 
#1 Camera 1 1 
#2 Vehicle 3 
#3 Headlights 15 
#4 Main beam 
#5 Low beam 

#6 Nighttime image ahead of vehicle 5 



[Figure 7] 

#1 Other light presence prohibition area 55 



IL Note - Please refer to numbered original for all figure legends. 
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[Figure 3] 

#1 Headlight-switching signal 
#2 CPU bus 

#3 Binary threshold value generation circuit 

#43x3 filter operation LSIs 

#53x3 filter data ROM 

#5 RAM for DSP 

#7 Binary threshold value signal 

#8 Comparator . . • 

#9 Image signal - 

#10 Binary image signal 

#11 Image bus 

#12 Binary image memory 



[Figure 5] 

#1 Binary conversion 
#2 Outline extraction 
#3 Light size computation 
#3b 

Horizontal length 

Vertical length 

#4 Symmetry extraction 

#5 

Symmei'y tracking 

a. Preceding vehicle 2 

b. Oncoming vehicle 2 

c. Preceding vehicle 1 

d. Oncoming vehicle 1 
#6 

Computation of distance to the vehicle 

a. Computes distance based on the width of the pair. 
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[Figure 4] 
(START) 

- I 

[Inputs an image.] - 601 

I 

[Converts the input image into binary data.] - 602 

I - 
[Extracts the outline of the image that has been converted into binary data.] - 603 

I 

[Computes light size.] - 604 
I 

[Extracts light symmetry.] - 605 

! 

[Tracks symmetry.] - 606 
I 

[Computes distance.] - 607 
I 

[Switches headlights.] - 608 
I 

(STOP) 

[Figure 6] 

(START) 
I 

[Uses the larger of the horizontal-direction length Lx or. the vertical-direction length Ly of the light as L] - 701 

[Sets the horizontal-direction search range to a value proportional to L. (Horizontal-direction search range = 
KL)] - 702 
I 

[Sets the vertical-direction search range to a certain value (e.g., 16 degrees)] - 703 

I 

[Extracts lights whose center positions are located inside the search range.] - 704 

I 

[Checks the horizontal-direction size. (KLxmax-Lx > Lx' > KLxminLx ] - 705 

r 

I 

[Checks the vertical-direction size. (KLymax Ly > Ly' > KLymin-Ly ] - 706 
I 

[Sets up other light presence prohibition area.] - 707 

[Checks whether or not the center point is located in the presence prohibition area.] - 708 
I 

[Extracts as a pair of lights.] - 709 



(STOP) 




c 



[Figure 10] 



#1 Distance to the vehicle [m] 

#2 Headlight . 

#3 Low beam 

#4 Main beam 

#5 Low beam 

#6 Preceding vehicle 

#7 Oncoming vehicle 

#8 Oncoming vehicle 

#9 Oncoming vehicle 

#10 Oncoming vehicle - 

Time [s] 



[Figure 8] 

(START) 
I 

[Extracts the characteristics of a pair.] - 801 
I 

[Computes the horizontal-direction search range in the next screen. Kx'-W] - 802 
I 

[Computes the vertical-direction search range in the next screen. Ky'-W] - 803 

[Corrects the search ranges in the next screen by taking up-down and left-right movements into 
consideration. Kx'-W + Ax and Ky'-W + Ay] — 804 

[Extracts a pair in the next screen, whose center position is located inside the search range in the next 
screen.] — 805 

[Extracts the characteristics of the pair whose center position is located inside the search range. ] - 806 

' I 

[Checks the size of the right light. KLrxmax-Lrx > Lrx' > KLrxmin-Lrx ] - 807 
I 

[Checks the size of the left light. KLrymax-Lry > Lry' > KLrymin-Lry] - 808 

I 

[Checks the slope of the pair. K0max>9 >&> KGrninB] - 809 
I 

[Selects a pair in the next screen.] - 810 



(STOP) 



[Figure 9] 

Ax: Left-right movements 
Ay: Up-down movements 
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